Hitherto, an image processing technique called relighting of calculating an illumination effect under an illumination environment different from that during image capture has been used in processing or the like of an image.
In relighting, subject information such as, the shape and reflection characteristics of a subject and illumination information such as, an illumination position and an illumination intensity which are associated with illumination of the subject are desired. These information items are the same as elements which are used in rendering of general three-dimensional computer graphics. In the shape of a subject, a method of the depth thereof is estimated by, for example, stereo matching using a plurality of cameras and restoring a model from sets of three-dimensional coordinates corresponding to each pixel is used. Regarding the reflection characteristics of the subject and the position of a light source, a method using the reflection of a light source in, for example, a spherical mirror is proposed.
In addition, in order to generate a suitable relighting image, for example, PTL 1 discloses a technique for automatically adjusting the intensity of a light source.